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Abstract 

Performance analysis of optimal signal detection using quantized 
received signals of a linear vector channel, which is an extension of 
code-division multiple-access (CDMA) or multiple-input multiple-output 
(MIMO) channels, in the large system limit, is presented in this paper. 
Here the dimensions of channel input and output are both sent to infinity 
while their ratio remains fixed. An optimal detector is one that uses a 
true channel model, true distribution of input signals, and perfect knowl- 
edge about quantization. Applying replica method developed in statistical 
mechanics, we show that, in the case of a noiseless channel, the optimal 
detector has perfect detection ability under certain conditions, and that 
for a noisy channel its detection ability decreases monotonically as the 
quantization step size increases. 



1 INTRODUCTION 

In modern wireless communication systems, many processes are performed on 
digital signals converted by an analog-to-digital (A/D) converter. In code- 
division multiple-access (CDMA) system which is one of key technologies for 
third generation cellular phone system, spreading, de-spreading, and code ac- 
quisition are all performed on digital signals. 

Although there have been many studies on performance analysis of digital 
communication systems, information-theoretic performance analyses usually as- 
sume that the received analog signals are available [1-3]. Previous studies [4,5], 



"This work was supported by the Grant-in- Aid for Scientific Research on Priority Areas 
(No. 18079010), the Ministry of Education, Culture, Sports, Science and Technology, Japan. 
This paper was presented in part at the IEICE Transactions on Fundamentals of Electronics, 
Communications and Computers (Japanese Edition) Vol. J91-A, No. 5, pp. 550-558, 2008. 



1 



considering the quantized effect of received signals, used non optimal signal 
detection schemes, such as linear filters. 

In this paper, we evaluate the performance of optimal signal detection using 
the quantized received signals of a linear vector channel, which is an extension of 
CDMA or multiple-input multiple-output (MIMO) channels, in the large system 
limit, where the dimensions of channel input and output are both sent to infinity 
while their ratio remains fixed. 



CHANNEL MODEL AND QUANTIZATION 
OF RECEIVED SIGNALS 



We consider a X-input, iV-output linear vector channel, defined as follows. 
Let Xo — (xqi, . . . , xqk) t € {+1, — 1} denote the channel input vector, and 
y = (j/i, . . . , yN) T denote the output vector, given a linear transform Hx$ of 
the inputs, where H is an N x K channel matrix. Assuming the additive white 
Gaussian (AWGN) channel (variance <7q), the outputs are described as 



y = Hx + a v, 



(1) 



where the components of v — (y\, vn) T follow the normal distribution 
Af(0, 1). To simplify the analysis, we assume a random channel matrix; the 
elements {H^k} are independent and identically distributed (i.i.d.) with mean 
zero and variance 1/N . 

In this study, we consider signal detection using received signals, which are 
quantized by an A/D converter at the receiver. We assume an A/D converter 
that quantizes received signal y into integer n, which satisfies the condition 
(n — l/2)d < y < (n + l/2)d, where d > is the quantization step size. The 
probability distribution of the quantized received signals n = (n\, . . . , tin) t , 
given the input xq and channel matrix H, is represented as 



P (n\Hx ) = Y[ Po{ n n\ H ^ x o), 
p (n M |Hja;o) 



(2) 
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where ffj, which denotes the /ith row of H and Q(x) — JJ° Dt, Dt = 

(27r) _1 / 2 e _t2 / 2 dt. Equation §5§ represents the input (xo)-output(n) character- 
istic of the quantization channel. 

The signal detection of channel input Xq, given the quantized received sig- 
nals n and the channel matrix H , can be solved by an inference scheme based on 
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Baysian inference. We define a prior as Po(xo), and assume Pq(xo) = 1/2 A '. The 
detector assumes the channel model of quantization channel to be P{n\Hx) = 
]l M =i Pi n n\Hn x ) an d the prior distribution to be P{x). The posterior distri- 
bution is represented as 



P(x\n, H) 



P(n\Hx)P{x) 
E x P(n\Hx)P(x)- 



(4) 



Maximizer of the Posterior Marginals (MPM) estimation it = 
argmax^ ^2 X{ k P(x\n, H) is the optimal inference scheme for minimiz- 
ing the component-wise estimation error probability, when the assumed channel 
model and prior distribution are matched to the true ones. Hereafter, we call 
the detector using true channel model and prior distribution as an optimal 
detector. 



3 PERFORMANCE ANALYSIS 
REPLICA METHOD 



USING 



We evaluate the performance of the optimal detector using the quantized re- 
ceived signals in the large system limit, where K, N — > oo while the ratio 
(3 = K/N is kept finite. This paper [6] studies the multiuser detection perfor- 
mance of a CDMA channel with an arbitrary memoryless channel and arbitrary 
distribution of channel inputs in the large system limit, using the replica method. 
Since the defined quantized channel ([3]) is an example of memoryless channels, 
one can apply the analysis reported in [6] to the performance evaluation of the 
optimal detector using quantized received signals in the large system limit. Al- 
though there is no rigorous justification for the replica method, we assume the 
validity of the replica method, and related techniques throughout this paper. 

In the large system limit, the estimation error probability of the optimal 
detector 



P, 



-xq , is, H 



1 K 
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(1 - x k x ok ) 



k=l 



(5) 



where Ex[- ••], which denotes the average with respect to X, can be evaluated 

as 

P b = Q(VE). (6) 
The parameter E is determined by solving the following equations for {m, E}, 

m = J tanh (e + VEz^ Dz, (7) 
E= ± f^g^, (8) 
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where po(n\^/ /3m) is defined as 




(9) 



and p' {n\A) = ^p (n\A). 

For some values of the parameters (<tq , d, f3), equations (J7J) and (jSJ) have three 
solutions, and these solutions can be distinguished by the value of estimation 
error probabilities P&; we define good, intermediate, and bad solutions from the 
smallest to the largest value of Pb- Furthermore, we define the correct solution 
as one minimizes the function T, which is defined as 

T = — ^2 J Po (n\y/(3mtj l°g Po (n\y/(3mtj Dt 



n— — oo 



-EM 



log 



2 cosh [E + s/Ez 



Dz. 



(10) 



It is known that the correct solution corresponds to the performance of the 
optimal detector. Since the exact evaluation of marginal posterior probability 
is usually computationally infeasible, we have to resort to approximation tech- 
niques. It is also known that a bad solution corresponds to the performance of 
the belief propagation based on iterative algorithm [7] , which produces approx- 
imate marginal posterior probabilities in low time complexity. 



4 RESULTS 

4.1 NOISELESS CHANNEL 

For a noiseless channel (cr = 0), the equations ([7]) and © have the perfect 
detection solution m = 1, E = oo, which provides Pb — for any values of 
d and (3. Figure [T] shows the bifurcation diagram. The optimal detector can 
achieve the perfect detection in the lower left side of the solid curve, and the 
coexistence of the solutions occurs above the broken curve. Figure [2] shows the 
estimation error probabilities of the optimal detector. 

4.2 NOISY CHANNEL 

For a noisy channel (<To > 0), the optimal detector cannot exhibit perfect de- 
tection, due to the effect of quantization noise. Figure [3] shows estimation error 
probabilities versus Eb/No = 1/(2ctq). The detection ability of the optimal 
detector decreases monotonically as the quantization step size increases. 
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Figure 1: Bifurcation diagram for the optimal detector over a noiseless channel. 
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Figure 2: Estimation error probabilities for the optimal detector over a noiseless 
channel. Solid lines with arrows represent the upper bound of dj ' \fp, for which 
the optimal detector can achieve the perfect detection. Broken lines with arrows 
represent the lower bounds of d/y/Ji, for which coexistence of the solutions 
occurs. 
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Figure 3: Estimation error probabilities versus Eij/Nq. (a) (3 — 1.0, (b)/3 = 1.8. 
The solid curves represent d/^ffi = 0.5(right), 0.25(left), and the broken curve 
represents d/y//3 — 0. The arrows represent the values of Eb/N , below which 
the bad solution becomes correct solution. 
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5 CONCLUSION 



In this paper, we have evaluated the performance of an optimal detector using 
the quantized received signals of a linear vector channel in the large system limit. 
Applying the replica method, we have shown that, for a noiseless channel, under 
certain conditions, the optimal detector has a perfect detection ability, i.e., it 
is not affected by quantization noise. We have also shown that, for a noisy 
channel, the optimal detector's detection ability decreases monotonically as the 
quantization step size increases. 
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